Computer-Implemented Method for Performing a System Assessment

ABSTRACT

A computer-implemented method including creating at least one run configuration package including at least a specification hierarchy, creating an application protocol interface including a generic code portion and an application-specific code portion, receiving an input for building an assessment environment from the run configuration package referencing an assessment application tool via the application protocol interface, and building the assessment environment based on the input. The assessment environment receives inputs using a standardized format, modifying various aspects of the assessment, based on the information in the specification hierarchy. The specification hierarchy is linked to a plurality of specification data objects for setting the assessment environment. The generic code portion is configured to provide a standardized instruction to the application-specific code portion and the application-specific code portion is configured to modify a format of the plurality of specification data objects linked to the specification hierarchy for the assessment application tool to be utilized.

TECHNICAL FIELD

The present disclosure relates to a computer-implemented method forperforming a system assessment, a computer program element forperforming a system assessment, a computer storage media and a computersystem including such computer storage media.

BACKGROUND

In engineering, computational analysis is carried out for a purpose ofpredicting performance or property of a technical system, whichtypically depends on many aspects such as an exact configuration of eachsubsystem of the technical system, different measurements to recordduring or after a simulation, precision required by the solver, initialand environmental conditions, and a precise load-case of the system.This may be reflected as a set-up or a configuration of the simulation.

Practically, the computational analysis requires setting up a numericalmodel of the technical system, populating it with suitable parameters,subjecting it to intended initial and boundary conditions, choosing asuitable solver system, deciding a hardware and evaluating results. Thismeans, each step of the simulation requires input or configuration dataand decisions. Traditionally, this information, i.e. configuration datahas been collected from wherever it originates, and inserted into acontext of the simulation, and the simulation is carried out. In case asubsequent analysis is requested, the corresponding entities need to bealtered, and the simulation is to be repeated.

There are simulation systems, which allow for a more modular approach ofthe subsystems, where a part of the configuration can be altered withoutchanging the entire data set. These systems are tailored for a specificsimulation domain, and generally split the configuration in parts thatmore or less match different physical domains.

SUMMARY

Hence, there may be a need to provide an improved method for preparingand executing a system assessment, which may optimize a utilization of awide variety of data.

The problem is at least partially solved or alleviated by the subjectmatter of the independent claims of the present disclosure, whereinfurther examples are incorporated in the dependent claims. It should benoted that the aspects of the disclosure described in the followingapply to the computer-implemented method for performing a systemassessment, the computer program element for performing a systemassessment, the non-transitory computer storage media and the computersystem including such non-transitory computer storage media.

The computer-implemented method includes creating at least one runconfiguration package including at least a specification hierarchy,creating an application protocol interface including a generic codeportion and an application-specific code portion, receiving an input forbuilding an assessment environment from the run configuration package inan assessment application tool via the application protocol interface,and building the assessment environment based on the input. Thespecification hierarchy is linked to a plurality of specification dataobjects for setting the assessment environment. The generic code portionis configured to provide a standardized instruction to theapplication-specific code portion and the application-specific codeportion is configured to modify a format of the plurality ofspecification data objects linked to the specification hierarchy for theassessment application tool to be utilized.

The computer-implemented method according to the present disclosure mayprovide a template for the system assessment, which allows astandardized configuration and/or execution of the system assessment.Accordingly, the system assessment may be optimized by loadingspecification data objects required for performing the system assessmentvia the template, which may allow an automation of the entire systemassessment. Thus, a saving of time, storage and/or processor capacitymay be achieved. Additionally, a user, who does not possess extensiveknowledge of the system assessment may easily accomplish the systemassessment.

To understand or predict a technical object or a technical system,computational analysis may be performed. For instance, the systemassessment may be applied for predicting and/or analyzing a crash eventof a vehicle, a driving performance of the vehicle, etc.

The run configuration package may be adapted to provide information forperforming the system assessment regarding the assessment applicationtool to be utilized, the assessment environment to be set up, system andparametric specifications to be applied in the system assessment,requirements for analyzing and reporting etc. The run configurationpackage may include such information by means of a mask, which may bestandardized and act as a global template for the system assessment.Accordingly, a user may be able to easily launch a system assessmentindependently of an object or a system to be simulated, an assessmenttool to be utilized, a test environment and/or test parameters.

The run configuration package may be generated automatically ormanually. The run configuration package may form a foundation forbuilding and executing the system assessment. The run configurationpackage may reference or indicate a particular work order forming thespecification hierarchy. In other words, the run configuration packagemay include at least a specification hierarchy, which may provide entiretechnical information necessary to perform the system assessment. Thetechnical information may define specifications such as a particularaspect of analysis including environment, system configuration, sensortechnology, test sequence, use cases, post-processing, reporting,experiment set-up, etc. contained in the plurality of specification dataobjects.

The specification hierarchy may be a specification organizationstructure, which may allow an establishment of a modulated set ofspecification data objects. The specification hierarchy may referenceand/or load the plurality of specification data objects. Thespecification hierarchy may include several information layers tofacilitate a transfer of specification data objects to the assessmentapplication tool.

Among the several information layers, only a top information layer ofthe specification hierarchy may communicate with the assessmentapplication tool for providing the specification data objects. However,the top information layer may only indicate the specification dataobjects required for setting and/or performing the assessmentenvironment but not store any specification data set. In other words,the top information layer may serve as an interface between theassessment application tool and data sets of the specification dataobjects.

Conventional simulation tools utilize generally monolithic data set forconfiguring and executing a simulation, which is often stored as asingle data file containing entire information. A different analysismay, thus, require another set of data, which may differ only in fewmodified parts. The unmodified parts of the new data set carry duplicateinformation without reference to each other between the contexts used.Apart from occupying more space, the duplicate information may not beseparately extracted from the monolith data set, therefore it is hard tokeep up to date. After some time, this can lead to a large number ofvariations of the monolith data set.

The application protocol interface may be a part of a process scheduler.The process scheduler may apply a multi-disciplinary design optimizationprocess. The application tool may be developed in Python, Perl, Ruby orthe like.

The process scheduler may include, for instance, modules for theexecution steps such as configuring, building, running, analyzing thesystem assessment, modules containing tool-specific sub-classes of theindividual steps, extending the functionality for each tool integration,a session handler sub-package for treating a state of an ongoingsimulation, examples for implemented environments, utilityfunctionality, handling format translations, mathematical conversions,package files for requirements, building and testing code,documentations etc.

Generally, to build an assessment environment and/or execute the systemassessment, an assessment application tool, in other words simulationtool such as MATLAB, ANSYS, Simpy, etc. may be applied. Each assessmentapplication tool, however, may require its own data format. Suchapplication-depending format is generally difficult to share betweendifferent assessment application tools. Hence, the data files may oftenhave to be adapted individually for the assessment application tool,which may lead to generating duplicate information demanding a largedata storage capacity.

Hence, the application protocol interface may be configured to interfacebetween the specification data objects to be collected to perform thesystem assessment and the data set of the assessment application tool.The application protocol interface may be divided by the generic codeportion and the application-specific code portion. In other words, theapplication protocol interface may be instantiated from the generic codeportion which may be extended by the application-specific code portion.

The generic code portion may be a tool agnostic part and configured fora generic specification interpretation. The generic code portion may bedefined by a modular code framework containing general code taking careof all settings that are generic regarding the user, the platform andthe system.

Whereas the application-specific code portion may be adapted for eachassessment application tool requirements, to take care of thetool-specific settings. Particularly, the application-specific codeportion may be configured to modify the format of the specification dataobjects with respect to the assessment application tool to be utilized.In other words, the application-specific code portion may include codesfor modifying the format of the specification data objects with respectto the assessment application tool to be utilized.

Hence, the specification data objects linked to the specificationhierarchy of a designated run configuration package may be interpretedby the generic code portion and their format may be converted to aformat compatible with the assessment application tool to be utilized.Accordingly, the application protocol interface may allow a unifiedinterface between the specification data objects and various assessmentapplication tools. As a result, the specification data objects may notneed to include the format for each individual application tool anduniversally applied in any assessment application tool.

The specification data objects including the application tool-specificdata may be transferred to the assessment application tool as an inputfor configuring the assessment environment. Thus, the assessmentapplication tool may be capable of setting up, or building, theassessment environment based on the input adapted and/or optimized tothe assessment application tool.

In an example, the run configuration package further includes processscheduler information. The process scheduler information of the runconfiguration package may provide, for instance, a specific version ofthe process scheduler to be utilized for performing the intended systemassessment.

The run configuration package further includes the specificationhierarchy. The specification hierarchy may provide, for instance, a nameof the assessment application tool to be utilized for performing thesystem assessment, its version, etc.

Accordingly, the run configuration package including the specificationhierarchy and the process scheduler information may provide entireconfiguration information necessary to perform the system assessment.Hence, any confusion relating to the process scheduler may be avoidedand a traceability of the process scheduler may be facilitated.

In an example, the specification hierarchy includes several informationlayers for modulating the plurality of specification data objects basedon a specification to be applied in a physical test environment. Forinstance, the specification hierarchy may include at least threeinformation layers. The first information layer may be configured toreference one or more specification data objects to be applied in theintended system assessment, wherein the specification data objects maybe classified based on a specification to be applied in a physical testenvironment. The first information layer may serve as a passage fortransferring the specification data objects to the assessmentapplication tool via the application protocol interface.

To the first information layer, a second information layer may be linkedfor loading the specification data objects referenced in the firstinformation layer. Further, to the second information layer, a thirdinformation layer may be linked for loading at least one, preferablyseveral data sets of the specification data objects loaded in the secondinformation layer.

Conventional simulation tools utilize generally a monolithic data setfor configuring and executing a simulation, which is often stored as asingle data file containing entire information. However, by defining theinformation hierarchy of the specification data objects to be sent tothe assessment application tool, the assessment application tool may notneed to access a stored monolithic data set for configuring andexecuting the simulation, which may improve process efficiency andreliability.

In an example, the method further includes storing the plurality ofspecification data objects as data records or posts in a database. In anexample, the method further includes storing the plurality ofspecification data objects as individual data files in a computerstorage media such as internal to a computing device, e.g. a computer'sSSD, or a removable device, e.g. an external HDD or universal serial bus(USB) flash drive.

In an example, the method further includes classifying the plurality ofspecification data objects by tool installation data, tool configurationdata, model parameters, test methods, excitation levels, pre- andpost-processing scripts and simulation settings. In other words, theplurality of specification data objects may be modulated with respect tothe real test environment, and each categorized specification dataobject may be individually selected for preparing the assessmentenvironment. The specification data objects may include specificationfiles. Thus, a quick access to the specification data objects and betterunderstanding of each specification data object may be achieved.

In an example, the method further includes installing the assessmentapplication tool based on the run configuration package. Particularly, aspecification data object defining the assessment application tool maybe referenced in the specification hierarchy of the run configurationpackage. The process scheduler may install the selected applicationassessment tool at a computer executing the system assessment, whereinthe computer may be a local computer and/or connected via a network.

In an example, the method may further include receiving the input forbuilding the simulation environment only from the run configurationpackage. The assessment application tool for performing the systemassessment may have only the access to the run configuration package.Specifically, the assessment application tool may be only connected tothe first, i.e. top information layer of the specification hierarchyreferencing one or more specification data objects necessary to buildthe assessment environment via the application protocol interface, whichmay modify the format of the referenced specification data objectaccording to the assessment application tool to be utilized.

In other words, the run configuration package may be the sole input tothe application tool and act as a “single source of truth”. Accordingly,the application tool may not need to access entire monolithic data setfor configuring and executing the simulation, which may improve processefficiency and reliability.

In an example, the method further includes executing the systemassessment based on the run configuration package. The process schedulerincluding execution steps of the system assessment may prompt theassessment application tool to run the simulation once the assessmentenvironment is built. Since such execution steps are defined in theprocess scheduler, the steps from installing the assessment applicationtool, building the assessment environment to executing the systemassessment may be performed automatically.

In an example, the method further includes modifying the assessmentenvironment using a standardized input template including parametricand/experimental information compiled during building the assessmentenvironment from the specification hierarchy. Accordingly, theassessment environment may be modified according to the purpose of theintended system assessment without recompling the assessmentenvironment. Thus, this may enable parametric studies and optimizationwhen performing the system assessment.

In an example, the method further includes referencing two or more runconfiguration packages in a Meta layer. The Meta layer may include atleast two run configuration packages, each of which define an individualsystem assessment. For instance, the two configuration packages maydiffer from each other in the assessment application tools, testobjects, particular specification requirements and/or parametricsetting.

To perform the system assessment at the Meta layer level, a Meta managerunit may be provided. The Meta manager unit may include, for instance,command-line interfacing, including help and instructions, runnerconfiguration, single runner configuration template builder includingdefault version of packages, package management including installing,verifying process scheduler, database manager including version controlinterfacing, runner, executing analyses through the process scheduler orthe like.

In an example, the method further includes executing the systemassessment of run configuration packages referenced in the Meta layer inparallel. The Meta layer may be particularly configured to perform twoor more system assessments simultaneously and/or consecutively, each ofwhich is defined in the respective run configuration package. The Metamanger unit may generate and/or provide an automated scheduling of theseveral system assessments, which may enable multiple batch runs on highperformance computing clusters.

In an example, the method further includes reading at least one outputof the system assessment and modifying the at least one output toinclude a predefined output format via the application protocolinterface. The process scheduler may also include codes forautomatically collecting output generated by the system assessment,wherein the output may include a specific data format of the respectiveassessment application tool.

Accordingly, the format of the output may be modified through theapplication protocol interface and the output may include a formatpredefined in the respective run configuration package, particularly inthe specification data objects. Thus, even though several systemassessments are performed by diverse assessment application tools fromdifferent engineering perspectives, their output may include a uniformformat, which may lead to an efficient combination of the computationalresults from the different engineering domains.

The same may also apply to the system assessment in the Meta layer thatthe outputs of individual system assessments with the respective runconfiguration packages may include the same format. Accordingly, adirect analysis and/or evaluation of the system assessments may beavailable.

In an example, the predefined output format is an applicationtool-independent format. The application protocol interface may beconfigured to modify the output of the system assessment, such that theoutput may include a format, which is independent of the applicationtool. In other words, the application protocol interface may be able tostandardize the output format. The output may be, for instance, a keyperformance indicator (KPI) referring to key figures that can be used tomeasure and/or determine a progress or a degree of fulfillment withregard to important objectives or critical success factors within asystem.

Accordingly, the Meta manager modifying, executing, and monitoring twoor more system assessments of run configuration packages may provide amulti-disciplinary assessment, i.e. Multi-disciplinary DesignOptimization.

In an example, the method further includes analyzing and reporting ofthe system assessment based on the output. A specification data objectof the run configuration package may also include a script forpost-processing, i.e. analyzing and/or reporting the output of thesystem assessment. The process scheduler may interpret suchspecification data object and run corresponding steps for analyzingand/or reporting. Since the output may include a standard format, theanalysis and/or evaluation of the outputs may be easily performed. Inthe reporting, the result may be represented as a table, a graphicand/or text. The result may compare and/or plot the output with theinput, e.g. run configuration package.

In an example, the method further includes version controlling the runconfiguration package. In an example, the method further includesversion controlling the Meta layer. The term “version controlling” maybe understood in that changes of data stored in the database may becontrolled and tracked. Each newly generated version may be saved in anarchive with a number or letter code. For instance, if a first change ismade in an individual data set, the changed data set may be identifiedby “ver. 2”.

Accordingly, any change of an initial or present data set of a runconfiguration package may initiate to create a new version of said dataset, which may cause, in turn, a version control of the specificationdata object and subsequently the run configuration package and the Metalayer. By updating and creating new version(s) of each of thespecification hierarchy, the run configuration package and the Metalayer, an evolution of information, normally reflected by its filename,may be easily tracked by the history itself and the hierarchy ofspecifications. Hence, traceability of the changes in the data setsand/or the specification data objects may be improved. As a result, theevolution of the simulation environments can be tracked uniquely.

According to the present disclosure, a computer program element forperforming a system assessment is presented. The computer programelement is adapted to perform the method steps as described above, whenbeing executed by a processing element.

According to the present disclosure, one or more computer storage mediais presented. The computer storage media is encoded with instructions,that when executed by a computer, cause the computer to perform theoperations of the respective method as described above.

The storage media may include internal to a computing device, such as acomputer's SSD, or a removable device such as an external HDD oruniversal serial bus (USB) flash drive. There are also other types ofstorage media, including magnetic tape, compact discs (CDs) andnon-volatile memory (NVM) cards.

According to the present disclosure, a computer system is presented. Thesystem includes one or more computer storage media as described aboveand a computer for executing the instructions.

It should be noted that the above examples may be combined with eachother irrespective of the aspect involved. Accordingly, the method maybe combined with structural features and, likewise, the system may becombined with features described above with regard to the method.

These and other examples of the present disclosure will become apparentfrom and elucidated with reference to the embodiments describedhereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS

Examples according to the present disclosure will be described in thefollowing with reference to the following drawings.

FIG. 1 shows schematically and exemplarily an example of a flowchart ofa computer-implemented method for performing a system assessmentaccording to the present disclosure.

FIG. 2 shows schematically and exemplarily an example of acomputer-implemented method for performing a system assessment accordingto the present disclosure.

FIG. 3 shows schematically and exemplarily an example of a specificationhierarchy according to the present disclosure.

DETAILED DESCRIPTION

FIG. 1 and FIG. 2 show a computer-implemented method for performing asystem assessment according to the present disclosure. The method may beadapted to understand or predict a technical object or a technicalsystem based on information defining the system to be analyzed and itsenvironment, for instance predicting and/or analyzing a crash event of avehicle, a driving performance of the vehicle, etc. The method may beperformed not necessarily in this order.

Each element in FIG. 1 in combination with FIG. 2 indicates:

-   -   S0 storing a plurality of specification data objects as data        records or posts in a database and/or as individual data files        in a computer storage media;    -   S1 creating a specification hierarchy 100 including several        information layers 20, 30, 40 for modulating the plurality of        specification data objects based on a specification to be        applied in a physical test environment;    -   S2 classifying the plurality of specification data objects by        tool installation data, tool configuration data, model        parameters, test methods, excitation levels, pre- and        post-processing scripts and simulation settings;    -   S3 creating at least one run configuration package 11,        -   wherein the run configuration package 11 the specification            hierarchy 100 and a process scheduler information 51, and            the process scheduler information 51 includes information of            the process scheduler 50 to be utilized for the system            assessment;    -   S31 version controlling the run configuration package 11;    -   S4 referencing two or more run configuration packages 11′, 11″        in a Meta layer 200;    -   S41 version controlling the Meta layer 200;    -   S5 opening a process scheduler 50 based on the run configuration        package 11;    -   S6 creating an application protocol interface 52 including a        generic code portion 53 and an application-specific code portion        54;        -   the generic code portion 53 is configured to provide a            standardized instruction to the application-specific code            portion 54        -   the application-specific code portion 54 is configured to            modify a format of the plurality of specification data            objects linked to the specification hierarchy 100 for an            assessment application tool 60 to be utilized;    -   S61 creating a standardized input template 80 based on the run        configuration package 11, particularly based on the parametric        and/or experimental specification data objects;    -   S7 installing the assessment application tool 60 based on the        run configuration package 11,        -   the application tool requires data 81 retrieved and/or            located to be accessible for functioning the application            tool;    -   S8 receiving an input for building an assessment environment 90        from the run configuration package 11 referencing the assessment        application tool 60 via the application protocol interface 52;    -   S9 building the assessment environment 90 based on the input;    -   S91 modifying the assessment environment 90 using the        standardized input template 80;    -   S10 executing the system assessment based on the run        configuration package 11;    -   S101 executing the system assessment of run configuration        packages 11, 11′, 11″ referenced in the Meta layer 200 in        parallel;    -   S11 reading at least one output 70 of the system assessment and        modifying the at least one output to include a predefined output        format via the application protocol interface 52,        -   wherein the predefined output 70 format being an application            tool-independent format;    -   S12 analyzing and reporting of the system assessment based on        the output 70.

Accordingly, a standardized approach for performing a system assessmentmay be achieved. The system assessment may be optimized by loadingspecification data objects required for the system assessment via atemplate, which may lead to a saving of time, storage and/or processorcapacity. This aspect-oriented approach enables a reuse and sharing ofrelevant specifications across domains and environments. In addition,the information provided by the specifications may be compatible withdifferent simulation tools and serve as a common information source.Additionally, a user, who does not possess extensive knowledge with thesystem assessment may be easily accomplish the system assessment.Further, a complete automation of the system assessment may be realized.

As shown in FIG. 3 , the information provided to perform the systemassessment may be constituted in a specification hierarchy 100 includingseveral information layers. At a top of the information hierarchy 100, afirst information layer 10, i.e. a run configuration package 11 isarranged. The first information layer 10 of the information hierarchy100 is, thus, linked to a plurality of specification data objects forsetting an assessment environment 90 and provides information of theapplication tool to be utilized for the system assessment. Following thefirst information layer 10, a second information layer 20 is arranged.The second information layer 20 includes at least a work order 21including a package of one or more specification data objects 31.

Following the second information layer 20, a third information layer 30is arranged. The third information layer 30 is configured to load eachspecification data object 31 referenced in the second information layer20. In other words, all of the specification data objects 31 indicatedin the second information layer 20 as necessary information for buildingthe assessment environment 90 are collected in the third informationlayer 30.

At the bottom of the information hierarchy 100, a fourth informationlayer 40 is arranged. The fourth information layer 40 is configured toload one or more data sets 41 of at least one specification data object31 loaded in the third information layer 30. Accordingly, theapplication tool for performing the system assessment may obtain fullinformation necessary to build the assessment environment 90.Additionally, the information hierarchy 100 may include furtherinformation layers providing information to the third information layer30.

In addition, the specification hierarchy 100 may further include Metalayer including two or more run configuration packages. An uppermostlayer of the specification hierarchy 100, which may be the Meta layer,the run configuration package 10 and/or the work order 20, may be thesole input to the application tool and act as a “single source oftruth”. Accordingly, the application tool may not need to access entiremonolithic data set for configuring and executing a simulation, whichmay improve process efficiency and reliability.

Since all of the first, second, third and fourth information layers 10,20, 30, 40 are configured to be version controlled individually, anevolution of the simulation environments can be tracked uniquely. Inparticular, by updating and creating new version(s) of each of thefirst, second and third information layers, the evolution ofinformation, normally reflected by its filename, may be easily trackedby the history itself and the hierarchy of specifications. As a result,traceability of the changes in the data sets and/or the specificationdata objects can be improved.

There are some examples of use cases of the computer-implemented methodaccording to the present disclosure:

Example 1

Specifying a simulation environment: Person AA develops a simulationmodel and publishes it in a model repository, a version controlleddatabase. AA defines also a simulation environment where this model isused by defining a) the “rig” setup specification describing the toolversion, its installation, execution parameters such as solver settingsetc., b) the “test object” setup specification pointing to all modelinformation such as repository paths etc., c) the “test method” setupspecification describing how the “test object” is run by the “test rig”,d) the “drivecases” setup specification that the “test method” consistsof, e) the “actuators” setup specification which describes how the “testobject” is excited in the specific “drivecases”, f) the “sensors” setupspecification defining all data to be collected, g) the “report” setupspecification defining what data to post-process, analyze and createreadable report from, h) the “parameters” setup specification describingwhich data in the “test” that can be varied. Person AA can nowautomatically configure the simulation environment for the specifiedcomputational platform, and execute the simulation using the batchautomation capabilities in the framework, meaning that no graphical userinterface is needed, removing the “clicking” a standard tool interfacerequires. The batch automation can be run multiple times where parameterchanges can be done for the different runs, enabling design ofexploration or optimization. Since it is easy to re-create thesimulation data, it can be deleted after the assessment is finished inorder to save space.

Example 2

Reusing information: Person BB wants to analyze another “test object” inthe same context as person AA has defined. This is simply done byreusing all information from example 1 except the “test object”specification that needs to be copied and modified to point to anothersimulation model. Then it is possible to make a similar assessment asperson AA has done by automatically configuring and executing theslightly modified test setup.

Example 3

Simplifying Hand-off: Person AA has planned a vacation, however, thework is not finished and need to be transferred to a colleague, personBB. Since person AA has defined the simulation environment with a set ofspecifications describing the content, it is very easy to re-create thisenvironment using the capability in the framework and continue the workfrom where person AA left.

Example 4

Sharing Computer Aided Engineering (CAE) Process: Person AA has beenasked to join a Multi-disciplinary Design Optimizations (MDO) projectbut has no time to participate. Instead, AA prepares an adequatesimulation environment and generate specifications for the automationframework and distributes them to the coordinator of the project who canimplement and execute the defined simulation within the MDO project.This is referred to as sharing expert knowledge to non-experts. When theMDO work has found promising design candidates, person AA will be askedto review them more thoroughly and validate them from the expert pointof view. This kind of sharing of CAE processes is also referred as CAEDemocratization.

Example 5

Checking simulation tool updates: When an update of the commercial toolis available, it is easy to check and validate its functionality byaltering the settings in the “rig” specification and rerunning referencesimulations. Thereby it is possible to track these kinds of changes moresystematically, and quality ensure the simulation environment in abetter way. If there is a need to re-assess historical simulations, justmodify the “rig” specifications of the concerned jobs, and rerun them.

Example 6

Trim generic settings: It may be possible to a) trim the actuators inthe test rig setup to get exactly the behavior wanted, b) alter thesolver settings to get the fastest simulation for a certain solvererror, or c) alter the sensitivity in signal filtering in thepost-process to trim the signal shape and thereby also the outputreadings from the simulation.

Example 7

Continuous Integration Chain: It is also possible to define automatedprocesses that are run by batch users. This ability to share awell-defined process is powerful and a fundamental requirement whendriving automated processes as in continuous integration (CI), wheretraceability and repeatability are crucial.

Example 8

Massive parallelization: When having a well-defined simulation processthat is both automatically configured and executed, massiveparallelization is possible. On each compatible computing node, thesimulation environment using one single source of information can betemporarily create and host. The processes are then injected bydifferent inputs, and the output is thereafter collected and managedtogether with the inputs in a master design of experiments table. Whenall simulations are done, the temporary simulation environments aredeleted.

Example 9

Parsing templates: A typical simulation consists of tool installation,configuration files, platform environment settings, user settings, etc.,in other words, a combination of binary and text files. To parameterizethe simulation, text files can be converted to template files includingkeywords instead of values. Thereby, the information can be altered bychanging the different parameters defined in a keyword list which ispopulated from the information defined in the modular specifications.This concept of changing parameters is implemented as the entry level ofan application protocol interface.

In the same way, it is possible to modify anything that is textual. If anew parameter is added to the model, the template would have to beupdated to work. When executing these kinds of simulation environments,it is popular to do it in batch mode using scripts. In the same manneras with the environment configuration, the execution script can becreated from a template and some keywords. Then the execution of thesimulation can be performed using single unified commands, despitedifferences in setup, and automate it using the information framework.This integration type is suitable for tools that can be pre-configuredbefore compilation and start of simulation.

Example 10

Offline tool interaction: Another way of interacting with a simulationtools, especially those that are encapsulated binaries, could be done bysystem commands defined by the tool developer. In such cases the defaultsimulation environment has to be launched to be able to modify it, forexample, an application engine that is running on a web server. Allcontent is stored in binary format, requiring direct programmaticalcontrol of the environment from the system command prompt. For example,in the following pseudo syntax “application-param vehicle_mass=2400”,the application with name “application” is instructed to change itsparameter “vehicle_mass”. In the same way, the simulation is executed bysending an appropriate instruction to the application, for example“application-run end_time=10, time_step=0.1”. The parameters within theinstructions are populated from the information defined in the modularspecifications, and these instructions are adapted to the specificapplications in the modular code package addressed for that specificenvironment.

Example 11

Online tool interaction: It is possible to incorporate methods foronline interaction, meaning that it is possible to programmaticallycontrol the process online, for example, during simulation requiringinternal states, changing excitation, stopping/starting execution,pausing, etc. This can be managed using a specific code package.

The computer-implemented method according to the present disclosure mayfacilitate thus, comparing new system development with an existingsystem. Any potential change from current set-ups may be extremelysimple to realize in simulation, at a very low cost/effort. Further, aneasy access to Multi-disciplinary Design Optimizations (MDO) studiesfocusing on the output and input rather than the process itself may beachieved. An advanced single disciplinary execution with many runs inparallel may be also possible by executing all relevant test objectssimultaneously. A simple massive parallelization for sensitivity studiesmay be achieved as well, due to the automated set-up and execution onall available compute nodes. In addition, a single shot execution fordebugging or detailed analysis, including Integrated DevelopmentEnvironment (IDE) or Graphical User Interface (GUI) interaction fordeveloping the analyses for different domains may be facilitated.

It has to be noted that examples of the disclosure are described withreference to different subject matters. In particular, some examples aredescribed with reference to method type claims whereas other examplesare described with reference to the device type claims. However, aperson skilled in the art will gather from the above and the followingdescription that, unless otherwise notified, in addition to anycombination of features belonging to one type of subject matter also anycombination between features relating to different subject matters isconsidered to be disclosed with this application. However, all featurescan be combined providing synergetic effects that are more than thesimple summation of the features.

While the disclosure has been illustrated and described in detail in thedrawings and description, such illustration and description are to beconsidered illustrative or exemplary and not restrictive. The disclosureis not limited to the disclosed examples. Other variations to thedisclosed examples can be understood and effected by those skilled inthe art in practicing a claimed disclosure, from a study of thedrawings, the disclosure, and the dependent claims.

In the claims, the word “comprising” does not exclude other elements orsteps, and the indefinite article “a” or “an” does not exclude aplurality. A single processor or other unit may fulfil the functions ofseveral items recited in the claims. The mere fact that certain measuresare recited in mutually different dependent claims does not indicatethat a combination of these measures cannot be used to advantage. Anyreference signs in the claims should not be construed as limiting thescope.

1. A computer-implemented method for performing a system assessmentcomprising: creating at least one run configuration package comprisingat least a specification hierarchy, the specification hierarchy beinglinked to a plurality of specification data objects for setting anassessment environment; creating an application protocol interfacecomprising a generic code portion and an application-specific codeportion, the generic code portion being configured to provide astandardized instruction to the application-specific code portion, theapplication-specific code portion being configured to modify a format ofthe plurality of specification data objects linked to the specificationhierarchy for an assessment application tool to be utilized; receivingan input for building the assessment environment from the runconfiguration package referencing the assessment application tool viathe application protocol interface; and building the assessmentenvironment based on the input.
 2. The computer-implemented methodaccording to claim 1, the run configuration package further comprisingprocess scheduler information.
 3. The computer-implemented methodaccording to claim 1, the specification hierarchy comprising severalinformation layers for modulating the plurality of specification dataobjects based on a specification to be applied in a physical testenvironment.
 4. The computer-implemented method according to claim 1,further comprising storing the plurality of specification data objectsas data records or posts in a database.
 5. The computer-implementedmethod according to claim 1, further comprising storing the plurality ofspecification data objects as individual data files in a computerstorage media.
 6. The computer-implemented method according to claim 1,further comprising classifying the plurality of specification dataobjects by tool installation data, tool configuration data, modelparameters, test methods, excitation levels, pre- and post-processingscripts and simulation settings.
 7. The computer-implemented methodaccording to claim 1, further comprising, receiving the input forbuilding the assessment environment only from the run configurationpackage.
 8. The computer-implemented method according to claim 1,further comprising, installing the assessment application tool based onthe run configuration package.
 9. The computer-implemented methodaccording to claim 1, further comprising, executing the systemassessment based on the run configuration package.
 10. Thecomputer-implemented method according to claim 1, further comprising,modifying the assessment environment using a standardized inputtemplate.
 11. The computer-implemented method according to claim 1,further comprising, referencing two or more run configuration packagesin a Meta layer.
 12. The computer-implemented method according to claim1, further comprising, executing the system assessment of runconfiguration packages referenced in the Meta layer in parallel.
 13. Thecomputer-implemented method according to claim 1, further comprisingreading at least one output of the system assessment and modifying theat least one output to comprise a predefined output format via theapplication protocol interface.
 14. The computer-implemented methodaccording to claim 13, the predefined output format being an applicationtool-independent format.
 15. The computer-implemented method accordingto claim 1, further comprising, analyzing and reporting of the systemassessment based on the output.
 16. The computer-implemented methodaccording to claim 1, further comprising, version controlling the runconfiguration package.
 17. The computer-implemented method according toclaim 1, further comprising, version controlling the Meta layer.
 18. Acomputer system comprising: a processor; memory storing instructionsexecuted by the processor to carry out method steps comprising: creatingat least one run configuration package comprising at least aspecification hierarchy, the specification hierarchy being linked to aplurality of specification data objects for setting an assessmentenvironment; creating an application protocol interface comprising ageneric code portion and an application-specific code portion, thegeneric code portion being configured to provide a standardizedinstruction to the application-specific code portion, theapplication-specific code portion being configured to modify a format ofthe plurality of specification data objects linked to the specificationhierarchy for an assessment application tool to be utilized; receivingan input for building the assessment environment from the runconfiguration package referencing the assessment application tool viathe application protocol interface; and building the assessmentenvironment based on the input.
 19. A non-transitory computer-readablemedium comprising instructions stored in a memory and executed by aprocessor to carry out the method steps comprising: creating at leastone run configuration package comprising at least a specificationhierarchy, the specification hierarchy being linked to a plurality ofspecification data objects for setting an assessment environment;creating an application protocol interface comprising a generic codeportion and an application-specific code portion, the generic codeportion being configured to provide a standardized instruction to theapplication-specific code portion, the application-specific code portionbeing configured to modify a format of the plurality of specificationdata objects linked to the specification hierarchy for an assessmentapplication tool to be utilized; receiving an input for building theassessment environment from the run configuration package referencingthe assessment application tool via the application protocol interface;and building the assessment environment based on the input.